One very important part of the software development process is software testing. Testing refers to the process of repetitively executing portions of a computer program to ensure that the program is functioning correctly and to identify errors within the executing program code. Testing is often performed by a human user of the computer. In addition to being time consuming and tedious, human verification can be error prone. The accuracy of the test is the sole responsibility of the tester. The results of such testing can be inaccurate and inconsistent since one tester may find a program to pass a given test while another tester may find the program to fail the same test, even though the test results are exactly the same in both cases. In addition to limitations in accuracy, manual testing also has time and speed limitations that make it difficult or impossible for the tester to assess the test results as they are presented in real-time.
However, because the testing process can be very repetitive, and therefore stressful on humans, automated testing programs have been developed that simulate human user input to a computer program. In particular, scripts can be generated for use with an automated testing program that simulates user input to the program to be tested in a particular order. For instance, a script may contain instructions for entering keystrokes, mouse movement and selection, and other types of user input commands. In order to thoroughly test a computer program, such scripts can be executed repetitively by the automated testing program on the program to be tested.
Software programs that are particularly susceptible to errors when being manually tested include applications that involve video processing (e.g., video composing, encoding, compression, decoding). For instance, one example of a decoding or rendering application is a media player. A media player typically processes multimedia data through a series of steps that include reading a data file, parsing the data into separate data streams (e.g., a video stream and audio stream), decoding each data stream, and rendering the data streams for presentation (e.g., on a video display or audio speaker). The various media that may be rendered by a media player may include, but is not limited to, audio, video, graphic, pictorial, text, and music, and any combination thereof. While media players may include specialized interfaces such as electronic devices (DVD and CD players, portable compressed audio players and other common devices), increasingly more media is being transmitted for viewing through browser interfaces for use on computers, personal digital assistants, Internet-enabled wireless telephones and the like. Popular forms of these media players include Windows Media Player by Microsoft, Quicktime by Apple, and RealOne players by RealNetworks. These media players, which may be generically referred to as rendering applications, are displayed through browsers such as Microsoft Windows, Netscape, Mozilla or other browsers.
The process of testing applications involving video or even audio processing media players can be particularly difficult to automate because the test results are generally presented as a visual image or as audio, which does do not lend themselves to easy quantification. That is, the results of such testing must be verified by visual or auditory inspection to ensure that the correct content is being presented and that it is being presented without distortion or any unrequested alterations. In addition, it is simply not feasible for a person to verify content at real-time video speeds (e.g., 24-30 frames/sec).
This Background is provided to introduce a brief context for the Summary and Detailed Description that follows. This Background is not intended to be an aid in determining the scope of the claimed subject matter nor be viewed as limiting the claimed subject matter to implementations that solve any or all of the disadvantages or problems presented above.